Camera for augmented reality display

ABSTRACT

An augmented reality display device includes a near-eye display configured to present imagery to a user eye. A camera is configured to capture light from a real-world environment and produce output useable to contribute to the imagery presented to the user eye via the near-eye display. The camera includes an aperture configured to receive the light from the real-world environment and an image sensor configured to respond to the light received from the real-world environment by generating sensor output signals useable to produce images on the near-eye display depicting the real-world environment. One or more optical elements provide an optical path for light from the aperture to the image sensor, the optical path having a length that is within a threshold of a distance between the user eye and the aperture of the camera.

BACKGROUND

“Augmented reality” systems typically present virtual images to a userwhile the user maintains at least some visibility of their surroundingenvironment. In this manner, computer-generated objects, characters, orother imagery may appear to the user as if they are integrated into theuser's real-world environment.

Some augmented reality devices present virtual images via a partially orfully transparent display of a head-mounted display device (HMD),allowing the user to directly view their surrounding environment whilethe virtual images are presented. Other augmented reality devicesincorporate fully opaque displays, which are used to providecomputer-generated images in conjunction with images of the real-worldcaptured by a camera. These are sometimes referred to as“video-passthrough” augmented reality devices.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 schematically illustrates use of an augmented reality displaydevice to view virtual imagery.

FIG. 2 schematically illustrates combining of images depicting areal-world environment with virtual images.

FIGS. 3A and 3B schematically depict an example camera for use with anaugmented reality display device.

FIG. 4 schematically depicts another example camera for use with anaugmented reality display device.

FIG. 5 schematically depicts another example camera for use with anaugmented reality display device.

FIG. 6 schematically depicts another example camera for use with anaugmented reality display device.

FIG. 7 schematically shows an example computing system.

DETAILED DESCRIPTION

In augmented reality scenarios, a common goal is to achievenear-seamless integration between virtual imagery and real-worldsurroundings. In other words, it is desirable for virtual objects to“blend-in” with their surroundings as much as is reasonably possible.For example, it is not desirable for virtual objects to appear to movein unnatural ways—e.g., “swimming” or “jittering.” It is similarly notdesirable for an augmented reality device to interfere with a user'sview of the real-world in distracting or disorienting ways, even in theabsence of any virtual imagery. Thus, in an ideal scenario, virtualobjects should be rendered and displayed for substantially the sameperspective from which the user views their real-world surroundings, andboth of these perspectives should ideally correspond to what the user'sbrain expects—i.e., the normal perspective of the user's eyes.

When these considerations are not met, augmented reality devices of alltypes have a propensity to cause motion sickness in some users. Motionsickness often occurs in situations in which the visual feedbackreceived by the human brain differs from the brain's perceived orexpected motion. In the specific case of video-passthrough augmentedreality devices, the camera(s) used to capture video of the real-worldenvironment for display to the user via the augmented reality displaydevice will inherently have a different perspective than the user's eyewould, if the user was viewing the real-world without the device. Thiscan cause the real-world to appear slightly magnified, and at adifferent perspective, as the camera is often spaced in front of theuser's eyes. This can also cause movements of the user's head to seemamplified—e.g., during nodding or head-turning motions. Furthermore,when virtual imagery is superimposed over the video of the real-world,the virtual imagery can appear to swim or jitter during head movements,as virtual imagery is typically rendered for a virtual pose that alsodiffers from the perspective of the camera. Each of these factors aloneand in combination can cause motion sickness in susceptible users.

Accordingly, the present disclosure is directed to a camera for use withaugmented reality display devices—e.g., those used to providevideo-passthrough augmented reality experiences. Specifically, thecamera described herein includes an aperture configured to receive lightfrom a real-world environment, and an image sensor configured to respondto the light by generating output signals useable to produce imagesdepicting the real-world environment on a near-eye display of anaugmented reality display device. The light from the real-worldenvironment follows an optical path from the aperture to the imagesensor that is defined by one or more optical elements within thecamera.

As one example, the optical elements may include two or more structuresconfigured to repeatedly reflect the light within the camera before itreaches the image sensor. In this manner, the optical path taken by thelight from the aperture to the image sensor may be longer than thephysical length of the camera. To mitigate motion sickness, the cameramay be constructed such that the length of the optical path taken by thelight is similar to (e.g., within a suitable threshold of) the distancebetween the user's eye and the aperture of the camera. Thus, imagescaptured by the camera may appear to have been captured from aperspective that is more consistent with the position of the user's eye,similar to a scenario in which the image sensor was disposed on or nearthe user's eye. This in turn can mitigate or alleviate symptoms ofmotion sickness in users of video-passthrough augmented reality devices.

It will be understood that, while the camera described herein isprimarily described in the context of video-passthrough augmentedreality, this is not limiting. Rather, a camera that uses one or moreoptical elements to alter the optical path of light from an aperture toan image sensor as described herein may be used in any suitableapplications. As non-limiting examples, the camera described herein maybe used for capturing video, still images (i.e., photography), orsensing, and may be used in conjunction with any virtual or augmentedreality devices and/or any other suitable computing devices. In someexamples, the camera described herein may be implemented as inputsubsystem 708 described below with respect to FIG. 7.

FIG. 1 schematically depicts an example augmented reality scenario.Specifically, FIG. 1 schematically shows a user 100 using an augmentedreality display device 102 in a real-world environment 104. Via anear-eye display 106 of the augmented reality display device, user 100has a field-of-view 108, corresponding to the portion of the user's viewthat is occluded by the near-eye display. The present disclosureprimarily contemplates a scenario in which the near-eye display isopaque. However, it will be understood that the approaches describedherein may also be applied to augmented reality display devices havingpartially or fully transparent near-eye displays. In some cases, atransparency of the near-eye display may be dynamically adjustable.

Augmented reality display device 102 provides an augmented realityexperience by presenting images to the user's eyes via near-eye display106. The images presented by the near-eye display include a mix ofimages depicting the real-world captured by a camera, as well as virtualimages generated by or viewed with the augmented reality display device.In the example of FIG. 1, the user can see a real couch 110 imaged bythe camera of the augmented reality display device, as well as a virtualcharacter 112 generated by the augmented reality display device andpresented on the near-eye display as part of an augmented realityexperience.

Additionally, or alternatively, virtual images may be generated by adifferent device from the augmented reality display device. For example,the augmented reality display device may receive pre-rendered virtualimages from a separate rendering computer and display the pre-renderedvirtual images via the near-eye display. The rendering computer may belocal to the augmented reality display device—e.g., communicating over awired connection or suitable wireless protocol—or the rendering computermay be remote from the augmented reality display device (e.g., a servercomputer), communicating over the Internet or other suitable network.

Mixing of real and virtual imagery is described in more detail withrespect to FIG. 2. Specifically, FIG. 2 schematically shows aspects ofaugmented reality display device 102 during use. Near eye display 106 ispositioned near a user eye 200 of user 100, such that images presentedon the near-eye display are visible to the user. Augmented realitydisplay device 102 is equipped with a camera 202 configured to capturelight from the real-world environment and generate output useable tocontribute to imagery presented to the user eye via the near-eyedisplay. Specifically, as shown, camera 202 captures an image 204 of thereal-world environment 104 depicted in FIG. 1 based on sensor outputsignals 203. The sensor output signals may take any suitable form—e.g.,color values for each of a plurality of pixels as measured by an imagesensor of the camera. Notably, image 204 may be a still image, or may beone frame of a video stream having any suitable framerate (e.g., 90frames-per-second).

The light received by the camera from the real-world environment willtypically be visible light, resulting in a visible-light image of thereal-world environment similar to what would be seen by the user eyewithout the augmented reality display device. In other examples,however, the camera may generate images of the real-world environmentbased on other spectra of light, such as infrared, ultraviolet, X-ray,etc. Because these spectra of light are not visible to human eyes, thecamera and/or augmented reality display device may convert pixel valuescorresponding to the received light into suitable visible RGB values.Furthermore, in FIG. 2, camera 202 is positioned along an optical axis212 extending away from user eye 200. In this manner, the position ofthe camera may be substantially similar to the position of the user eyerelative to an X axis (extending into the page) and a Y axis (extendingvertically), only differing according to a Z axis (extendinghorizontally). In other examples, however, the camera need not lie alongthe optical axis extending away from the eye, but rather may have anysuitable position relative to the X, Y, and Z axes. More detailsregarding camera 202 will be provided below with respect to FIGS. 3-6.

Augmented reality display device 102 also includes a virtual imagerenderer 206 configured to generate virtual images for display to theuser eye via the near-eye display. As shown, the virtual image rendererhas generated a virtual image 208 depicting a fictional wizardcharacter. The virtual image 208 may then be superimposed over thereal-world image 204 to generate an augmented reality image 210, whichis presented on the near-eye display for viewing by the user eye. Inthis manner, the user may perceive the fictional wizard character as ifit were physically present in the user's real-world environment.

Virtual imagery may be generated in any suitable way and for anysuitable purpose. Virtual images may include any suitable image content,including virtual objects, characters, interface elements,heads-up-display (HUD) content, lighting effects, text,highlighting/shading, etc. For the sake of simplicity, all virtual imagecontent included in a virtual image will be referred to as “virtualobjects.”

Typically, a virtual image will include one or more virtual objects thatare rendered such that they appear to have a three-dimensional positionor “pose” with respect to a surrounding real or virtual environment. Insome examples, the pose may be a six degree-of-freedom (6DOF) pose,although fewer than six degrees-of-freedom may be used. Virtual objectsmay be “world-locked,” such that they appear to maintain their positionsrelative to the surrounding environment even as the user's perspectivechanges. Additionally, or alternatively, virtual objects may be“body-locked,” such that they appear to maintain a fixed positionrelative to the user as the user's perspective changes. Body-lockedand/or world-locked virtual objects may additionally move along theirown paths that are defined relative to either the user or theworld—e.g., a body-locked virtual object may constantly appear to circlea user's head regardless of the user's position, and a world-lockedvirtual object may continuously float around a real-world roomregardless of the user's position.

In any case, virtual images will be rendered relative to a virtualcamera pose, corresponding to the approximate viewpoint of the user. Forexample, virtual images may be rendered relative to the approximatecenter of the augmented reality display device, as that may roughlycorrespond to the positions of the user's eyes while the augmentedreality display device is worn. As the pose of the augmented realitydisplay device changes—e.g., due to user movements—rendering of virtualimages may be updated to maintain the body-locked or world-lockedorientations of the virtual objects. The pose of the augmented realitydisplay device may be tracked in any suitable way. In some examples, theaugmented reality display device may include one or more trackingcameras (not shown) configured to track the pose of the augmentedreality display device by imaging the real-world environment.Additionally, or alternatively, the augmented reality display device mayinclude an inertial measurement unit (IMU) equipped with one or moresuitable motion sensors configured to measure changes in the augmentedreality display device's pose. Such motion sensors can include, asexamples, gyroscopes, accelerometers, and magnetometers.

Virtual images may be updated with any suitable frame rate. In someexamples, the frame rate at which virtual images are presented may matchthe frame rate at which images of the real-world environment arecaptured—e.g., 90 frames-per-second—although other frame rates mayalternatively be used. Virtual images may be rendered by any suitableprocessing or logic componentry of the augmented reality display device.In other words, virtual image renderer 206 may be implemented as anysuitable computer processor, or other component suitable for generatingvirtual images. In some examples, virtual image renderer 206 may beimplemented as logic subsystem 702 described below with respect to FIG.7. Furthermore, augmented reality display device 102 may additionally oralternatively include other computer components not depicted in FIG. 2and not explicitly described herein, configured to provide othercomputer functions of the augmented reality display device. In someexamples, augmented reality display device 102 may be implemented ascomputing system 700 described below with respect to FIG. 7.

As discussed above, camera 202 captures an image 204 of the real-worldenvironment, which is combined with virtual image 208 to give augmentedreality image 210. Augmented reality image 210 is presented to user eye200 via near-eye display 106, substantially replacing at least a portionof the user's view of their surrounding environment. However, as shown,the position of the camera is different from the position of the usereye, meaning the real-world is imaged from a different perspective thanthe user's brain would ordinarily expect. As discussed above, this cancause a variety of issues that can contribute to motion sickness, suchas magnification of the real-world, amplification of head motions,swimming or jittering of virtual imagery, etc.

This can be mitigated or alleviated when the optical path taken by thelight from the aperture of the camera to the image sensor within thecamera is modified to have a length that is within a suitable thresholdof the distance between the user's eye and the camera's aperture. Thishas a similar effect to physically positioning the image sensor closerto the actual position of the user's eye.

Accordingly, FIGS. 3A and 3B schematically depict details of opticalelements disposed within camera 202 that may serve to lengthen theoptical path taken by light from the aperture to the image sensor.Specifically, FIG. 3A schematically depicts user eye 200, as well asnear-eye display 106 and camera 202 of augmented reality display device102, shown in cross-section. Camera 202 includes an aperture 300configured to receive light rays 302A and 302B from a real-worldenvironment. Camera 202 also includes an image sensor 304 configured torespond to the light received from the real-world environment bygenerating sensor output signals useable to produce images on thenear-eye display depicting the real-world environment (e.g., image 204shown in FIG. 2). Light rays 302A and 302B follow an optical path fromthe aperture to the image sensor provided by one or more opticalelements of the camera, as will be described in more detail below.

The camera aperture may have any suitable shape and size. In thisexample, the aperture includes an annular ring formed in anenvironment-facing surface of the camera. This is shown in more detailin FIG. 6, which shows an environment-facing surface of camera 202. Asshown, camera 202 includes a single annular aperture formed near anoutside edge of the camera. Image sensor 304 is shown in dashed lines,illustrating the relative position of the image sensor as compared tothe body and aperture of the camera. In other examples, however, thecamera may include multiple apertures—e.g., multiple annular ringshaving different radii—and/or apertures having other suitable shapes.

Any suitable type of image sensor may be used. In one example, the imagesensor may be a complementary metal-oxide-semiconductor (CMOS) sensor.As another example, the image sensor could be a charge-coupled device(CCD) sensor.

Returning to FIG. 3A, camera 202 includes two light-redirection surfaces306A and 306B that provide the optical path taken by the light from theaperture to the image sensor. Specifically, light rays 302A and 302B arerepeatedly reflected between the two light-redirection surfaces, therebylengthening the optical path. It will be understood that the specificarrangement depicted in FIG. 3A is provided for the sake of example andis not limiting. For example, in FIG. 3A, the light is reflected by theaperture-distal light-redirection surface two times along the opticalpath, and reflected by the aperture-proximal light-redirection surfaceonce. In other implementations, however, light may be reflected by eachof the light-redirection surfaces any suitable number of times along theoptical path. This may vary based on size constraints, materialconstraints, a desired length of the optical path, and/or otherconsiderations.

Furthermore, in FIG. 3A, the image sensor is disposed on anaperture-proximal light-redirection surface of the two light-redirectionsurfaces. Once again, however, this is not limiting. In other examples,the image sensor may be disposed on the aperture-distallight-redirection surface, or have any other suitable position withrespect to other components of camera 202.

As discussed above, symptoms of motion sickness may be mitigated orentirely alleviated when the optical path taken by light through thecamera has a length that is within a suitable threshold of a distancebetween the user eye and the aperture of the camera. This is illustratedin FIGS. 3A and 3B. Notably, the repeated reflections of light betweenlight-redirection surfaces 306A and 306B increases the length of theoptical path to be greater than the physical length of the camera, andsubstantially equal to the distance between user eye 200 and aperture300.

FIG. 3B shows an equivalent “unfolded” view of camera 202. Specifically,user eye 200, aperture 300, light rays 302A and 302B, andaperture-proximal light-redirection surface 306A are still shown, whileother elements of camera 202 are omitted. In essence, FIG. 3Billustrates an alternate optical path taken by the light, in which thelight travels the same distance through space but is not repeatedlyreflected by the light-redirection surfaces. Reference lines 312indicate the positions along the “unfolded” optical path at which thelight is reflected in FIG. 3A. By following this alternate “unfolded”path, the light rays converge at the position of the user eye ratherthan the image sensor. This is equivalent to a scenario in which theimage sensor is disposed on or near the user's eye, as indicated by box314 showing the equivalent position of the image sensor. In other words,the configuration depicted in FIG. 3A allows the image sensor to imagethe real-world environment as if the image sensor was located at box 314shown in FIG. 3B. Thus, the image sensor will have a similar perspectiveto the user eye, without requiring eye surgery or interfering with eyefunctions, as would be the case if the image sensor was actually locatedat box 314.

It will be understood that the length of the optical path taken by thelight may have any suitable relationship to the distance between theuser eye and camera aperture. It is generally desirable for the lengthof the optical path taken by the light to be as close as possible to thedistance between the user eye and the camera. However, it is estimatedthat significant benefits may be achieved even if the length of theoptical path is only 50% of the distance between the user eye and thecamera aperture. Thus, in general, the length of the optical path willbe within a suitable threshold of the distance between the user eye andcamera aperture. As one example, the threshold may be equal to 50%. Inother examples, the threshold employed may be 25% or 10%. Any suitablevalue may be used, and in some cases, the threshold may be based on thedistance between the eyes and the objects being viewed. In some cases,the length of the optical path taken by the light may bedynamically-adjustable, as will be described in more detail below.

In the example of FIG. 3A, the two light-redirection surfaces areindependent components separated by an airgap. In general, the twolight-redirection surfaces may be separated by any suitableoptically-transmissive medium. Alternatively, the two light-redirectionsurfaces may be first and second surfaces of a single structure. This isschematically illustrated in FIG. 4, which shows a different examplecamera 400 having an aperture 402, which receives light rays 404A and404B. The light rays follow an optical path from the aperture to animage sensor 406, in which the light rays are repeatedly reflected bylight-redirection surfaces 408A and 408B. Unlike FIG. 3A, in thisexample, the light-redirection surfaces are first and second surfaces ofa single optically-transmissive substrate 410.

In cases where the two light-redirection surfaces are physicallyseparate, as in FIG. 3A, the distance between the two light-redirectionsurfaces may be dynamically adjustable. In other words, either or bothof the two light-redirection surfaces may be movable relative to thecamera body to change the distance between the two surfaces. To thisend, FIG. 3A includes a distance adjuster 308 between the twolight-redirection surfaces operatively coupled to a controller 310. Asone example, either or both of the light-redirection surfaces may beattached to a rail, and the distance adjuster may include a motorconfigured to move the light-redirection surface along the rail—e.g.,relative to the Z axis. Controller 310 may include any suitablecollection of computer hardware and firmware components, such that thecontroller may be activated by the augmented reality display device(and/or by the user) to dynamically adjust the distance between thelight-redirection surfaces. By adjusting this difference, the length ofthe optical path taken by the light from the aperture to the imagesensor may be changed. In some scenarios, adjusting this distance maybenefit users whose eye sockets are different depths. This can be usedto focus the light received from the real-world environment to a desiredoptical power—e.g., to enable autofocus and/or to fine-tune thereal-world image presented to the user to mitigate symptoms of motionsickness.

Similar effects may be achieved when the two light-redirection surfacesare first and second surfaces of a single optically-transmissivesubstrate, as is the case in FIG. 4. For example, theoptically-transmissive substrate may be an electro-optic material havinga dynamically-changeable refractive index, such as a liquid crystal orother suitable material. Thus, as with FIG. 3A, camera 400 may beoperatively coupled to a controller configured to affect the lighttransmissive properties of the optically-transmissive substrate bydynamically supplying an electrical voltage or current.

In the example of FIG. 3A, the depicted light-redirection surfaces areplanar and parallel to one another. In other examples, however, thisneed not be the case. FIG. 5 depicts another example camera 500. As withcameras 202 and 400, camera 500 includes an aperture 502 configured toreceive light rays 504A and 504B from a real-world environment. Thelight rays follow an optical path from the aperture to an image sensor506, in which the light rays are repeatedly reflected bylight-redirection surfaces 508A and 508B. Unlike the earlier-describedcameras, the light-redirection surfaces in camera 500 are curved. Suchcurved light-redirection surfaces may have any suitable curvatureradius, and the curves may be convex or concave depending on theimplementation. Furthermore, each of the two light-redirection surfacesneed not be curved in the same manner or to the same extent.

Light-redirection surfaces 508A and 508B are independent componentsseparated by an airgap. Thus, as with camera 202, the distance betweenthe curved light-redirection surfaces may in some cases bedynamically-adjustable via any suitable mechanism. Alternatively, aswith camera 400, curved light-redirection surfaces may be separatesurfaces of a single optically-transmissive substrate. In such cases,the optically-transmissive substrate may in some cases be anelectro-optic material having a dynamically-adjustable refractive index.

Light-redirection surfaces, as well as other components of the camerasdescribed herein, may be produced via any suitable fabrication methods.As one example, when planar light-redirection surfaces are used, suchsurfaces may be produced via wafer-scale fabrication. Thus, thelight-redirection surfaces may be wafers of silicon, or another suitablematerial, with one or more optical elements or coatings applied to thewafer. Alternatively, the light-redirection surfaces may be subdivisionsof a single wafer. When the light-redirection surfaces are curved,another suitable fabrication method may be used, such as diamondturning.

In general, light-redirection surfaces, as well as other opticalcomponents of the cameras described herein, may be composed of anysuitable materials. As examples, light-redirection surfaces may becomposed of silicon, plastic, metal, glass, ceramics, or other suitablematerials. When the light-redirection surfaces are separate surfaces ofan optically-transmissive substrate, the substrate may be constructedfrom any suitably-transmissive materials—e.g., glass, transparentplastic, transparent silicon, or electro-optic materials. In variouscases, substantially the entire surface area of each light-redirectionsurface may be reflective, or only individual portions of thelight-redirection surfaces may be reflective. In other words, individualportions of each light-redirection surface may have reflective coatingsprinted on, or otherwise applied to, the surface.

Furthermore, optical elements disposed within the cameras described bythis disclosure need not be limited to the light-redirection surfacesdescribed thus far. Rather, as non-limiting examples, suitable opticalelements can include various reflective elements, transmissive elements,refractive elements, diffractive elements, holographic elements, lenses,filters, diffusers, etc. One or more lenses may be disposed at or nearthe aperture of the camera, the image sensor of the camera, on one orboth of the light-redirection surfaces, or other suitable positionswithin the camera to focus or otherwise redirect the light. Either orboth of the light-redirection surfaces may include Fresnel elements.Suitable filters may be used to, for example, filter certain wavelengthsof light, filter different polarizations of light, filter light based onspatial frequency (e.g., high-frequency patterns may be filtered out),filter out curved/straight elements, etc. It will be understood that theexact optical elements included with any particular camera can varywildly depending on the intended application for the camera, and suchoptical elements can include virtually any suitable current or futureoptical technologies.

The methods and processes described herein may be tied to a computingsystem of one or more computing devices. In particular, such methods andprocesses may be implemented as an executable computer-applicationprogram, a network-accessible computing service, anapplication-programming interface (API), a library, or a combination ofthe above and/or other compute resources.

FIG. 7 schematically shows a simplified representation of a computingsystem 700 configured to provide any to all of the compute functionalitydescribed herein. Computing system 700 may take the form of one or morepersonal computers, network-accessible server computers, tabletcomputers, home-entertainment computers, gaming devices, mobilecomputing devices, mobile communication devices (e.g., smart phone),virtual/augmented/mixed reality computing devices, wearable computingdevices, Internet of Things (IoT) devices, embedded computing devices,and/or other computing devices.

Computing system 700 includes a logic subsystem 702 and a storagesubsystem 704. Computing system 700 may optionally include a displaysubsystem 706, input subsystem 708, communication subsystem 710, and/orother subsystems not shown in FIG. 7.

Logic subsystem 702 includes one or more physical devices configured toexecute instructions. For example, the logic subsystem may be configuredto execute instructions that are part of one or more applications,services, or other logical constructs. The logic subsystem may includeone or more hardware processors configured to execute softwareinstructions. Additionally, or alternatively, the logic subsystem mayinclude one or more hardware or firmware devices configured to executehardware or firmware instructions. Processors of the logic subsystem maybe single-core or multi-core, and the instructions executed thereon maybe configured for sequential, parallel, and/or distributed processing.Individual components of the logic subsystem optionally may bedistributed among two or more separate devices, which may be remotelylocated and/or configured for coordinated processing. Aspects of thelogic subsystem may be virtualized and executed by remotely-accessible,networked computing devices configured in a cloud-computingconfiguration.

Storage subsystem 704 includes one or more physical devices configuredto temporarily and/or permanently hold computer information such as dataand instructions executable by the logic subsystem. When the storagesubsystem includes two or more devices, the devices may be collocatedand/or remotely located. Storage subsystem 704 may include volatile,nonvolatile, dynamic, static, read/write, read-only, random-access,sequential-access, location-addressable, file-addressable, and/orcontent-addressable devices. Storage subsystem 704 may include removableand/or built-in devices. When the logic subsystem executes instructions,the state of storage subsystem 704 may be transformed—e.g., to holddifferent data.

Aspects of logic subsystem 702 and storage subsystem 704 may beintegrated together into one or more hardware-logic components. Suchhardware-logic components may include program- and application-specificintegrated circuits (PASIC/ASICs), program- and application-specificstandard products (PSSP/ASSPs), system-on-a-chip (SOC), and complexprogrammable logic devices (CPLDs), for example.

The logic subsystem and the storage subsystem may cooperate toinstantiate one or more logic machines. As used herein, the term“machine” is used to collectively refer to the combination of hardware,firmware, software, instructions, and/or any other componentscooperating to provide computer functionality. In other words,“machines” are never abstract ideas and always have a tangible form. Amachine may be instantiated by a single computing device, or a machinemay include two or more sub-components instantiated by two or moredifferent computing devices. In some implementations a machine includesa local component (e.g., software application executed by a computerprocessor) cooperating with a remote component (e.g., cloud computingservice provided by a network of server computers). The software and/orother instructions that give a particular machine its functionality mayoptionally be saved as one or more unexecuted modules on one or moresuitable storage devices.

When included, display subsystem 706 may be used to present a visualrepresentation of data held by storage subsystem 704. This visualrepresentation may take the form of a graphical user interface (GUI).Display subsystem 706 may include one or more display devices utilizingvirtually any type of technology. In some implementations, displaysubsystem may include one or more virtual-, augmented-, or mixed realitydisplays.

When included, input subsystem 708 may comprise or interface with one ormore input devices. An input device may include a sensor device or auser input device. Examples of user input devices include a keyboard,mouse, touch screen, or game controller. In some embodiments, the inputsubsystem may comprise or interface with selected natural user input(NUI) componentry. Such componentry may be integrated or peripheral, andthe transduction and/or processing of input actions may be handled on-or off-board. Example NUI componentry may include a microphone forspeech and/or voice recognition; an infrared, color, stereoscopic,and/or depth camera for machine vision and/or gesture recognition; ahead tracker, eye tracker, accelerometer, and/or gyroscope for motiondetection and/or intent recognition.

When included, communication subsystem 710 may be configured tocommunicatively couple computing system 700 with one or more othercomputing devices. Communication subsystem 710 may include wired and/orwireless communication devices compatible with one or more differentcommunication protocols. The communication subsystem may be configuredfor communication via personal-, local- and/or wide-area networks.

This disclosure is presented by way of example and with reference to theassociated drawing figures. Components, process steps, and otherelements that may be substantially the same in one or more of thefigures are identified coordinately and are described with minimalrepetition. It will be noted, however, that elements identifiedcoordinately may also differ to some degree. It will be further notedthat some figures may be schematic and not drawn to scale. The variousdrawing scales, aspect ratios, and numbers of components shown in thefigures may be purposely distorted to make certain features orrelationships easier to see.

In an example, an augmented reality display device comprises: a near-eyedisplay configured to present imagery to a user eye; and a cameraconfigured to capture light from a real-world environment and produceoutput useable to contribute to the imagery presented to the user eyevia the near-eye display, the camera comprising: an aperture configuredto receive the light from the real-world environment; an image sensorconfigured to respond to the light received from the real-worldenvironment by generating sensor output signals useable to produceimages on the near-eye display depicting the real-world environment; andone or more optical elements providing an optical path for light fromthe aperture to the image sensor, the optical path having a length thatis within a threshold of a distance between the user eye and theaperture of the camera. In this example or any other example, the one ormore optical elements include two light-redirection surfaces configuredto provide the optical path by repeatedly reflecting light between thetwo light-redirection surfaces. In this example or any other example,the image sensor is disposed on an aperture-proximal light-redirectionsurface of the two light-redirection surfaces. In this example or anyother example, light is reflected by an aperture-distallight-redirection surface of the two light-redirection surfaces two ormore times along the optical path from the aperture to the image sensor.In this example or any other example, the two light-redirection surfacesare independent components separated by an airgap. In this example orany other example, a distance between the two light-redirection surfacesis dynamically adjustable. In this example or any other example, the twolight-redirection surfaces are first and second surfaces of a singleoptically transmissive substrate. In this example or any other example,the optically transmissive substrate is an electro-optic material havinga dynamically-changeable refractive index. In this example or any otherexample, the two light-redirection surfaces are planar and parallel toone another. In this example or any other example, the twolight-redirection surfaces are produced via wafer-scale fabrication. Inthis example or any other example, the two light-redirection surfacesare curved. In this example or any other example, the twolight-redirection surfaces are produced via diamond turning. In thisexample or any other example, one or both of the two light-redirectionsurfaces include Fresnel elements. In this example or any other example,the aperture includes one or more annular rings formed in anenvironment-facing surface of the camera. In this example or any otherexample, the camera is positioned along an optical axis extending awayfrom the user eye. In this example or any other example, the augmentedreality display device further comprises a virtual image rendererconfigured to generate virtual images for display to the user eye viathe near-eye display, where the virtual images are superimposed over theimages depicting the real-world environment. In this example or anyother example, the threshold is 50%. In this example or any otherexample, the threshold is 10% percent.

In an example, an augmented reality display device comprises: a near-eyedisplay configured to present imagery to a user eye; a virtual imagerenderer configured to generate virtual images that contribute to theimagery presented to the user eye via the near-eye display; and a cameraconfigured to capture light from a real-world environment and produceoutput useable to contribute to the imagery presented to the user eyevia the near-eye display, the camera comprising: an aperture configuredto receive the light from the real-world environment; an image sensorconfigured to respond to the light received from the real-worldenvironment by generating sensor output signals useable to produceimages on the near-eye display depicting the real-world environment; andtwo or more light-redirection surfaces configured to repeatedly reflectlight between the two or more light-redirection surfaces, therebyproviding an optical path for light from the aperture to the imagesensor, the optical path having a length that is within a threshold of adistance between the user eye and the aperture of the camera.

In an example, a camera comprises: an aperture configured to receivelight from a real-world environment, the aperture including an annularring formed in an environment-facing surface of the camera; an imagesensor configured to respond to the light received from the real-worldenvironment by generating sensor output signals useable to produceimages for display to a user eye; and two or more light-redirectionsurfaces configured to repeatedly reflect light between the two or morelight-redirection surfaces, thereby providing an optical path for lightfrom the aperture to the image sensor, the optical path having a lengththat is within a threshold of a distance between the user eye and theaperture of the camera.

It will be understood that the configurations and/or approachesdescribed herein are exemplary in nature, and that these specificembodiments or examples are not to be considered in a limiting sense,because numerous variations are possible. The specific routines ormethods described herein may represent one or more of any number ofprocessing strategies. As such, various acts illustrated and/ordescribed may be performed in the sequence illustrated and/or described,in other sequences, in parallel, or omitted. Likewise, the order of theabove-described processes may be changed.

The invention claimed is:
 1. An augmented reality display device,comprising: a near-eye display configured to present imagery to a usereye; and a camera configured to capture light from a real-worldenvironment and produce output useable to contribute to the imagerypresented to the user eye via the near-eye display, the cameracomprising: an aperture configured to receive the light from thereal-world environment; an image sensor configured to respond to thelight received from the real-world environment by generating sensoroutput signals useable to produce images on the near-eye displaydepicting the real-world environment; and one or more optical elementsproviding an optical path for light from the aperture to the imagesensor, the optical path having a length that is within a threshold of adistance between the user eye and the aperture of the camera.
 2. Theaugmented reality display device of claim 1, where the one or moreoptical elements include two light-redirection surfaces configured toprovide the optical path by repeatedly reflecting light between the twolight-redirection surfaces.
 3. The augmented reality display device ofclaim 2, where the image sensor is disposed on an aperture-proximallight-redirection surface of the two light-redirection surfaces.
 4. Theaugmented reality display device of claim 2, where light is reflected byan aperture-distal light-redirection surface of the twolight-redirection surfaces two or more times along the optical path fromthe aperture to the image sensor.
 5. The augmented reality displaydevice of claim 2, where the two light-redirection surfaces areindependent components separated by an airgap.
 6. The augmented realitydisplay device of claim 5, where a distance between the twolight-redirection surfaces is dynamically adjustable.
 7. The augmentedreality display device of claim 2, where the two light-redirectionsurfaces are first and second surfaces of a single opticallytransmissive substrate.
 8. The augmented reality display device of claim7, where the optically transmissive substrate is an electro-opticmaterial having a dynamically-changeable refractive index.
 9. Theaugmented reality display device of claim 2, where the twolight-redirection surfaces are planar and parallel to one another. 10.The augmented reality display device of claim 9, where the twolight-redirection surfaces are produced via wafer-scale fabrication. 11.The augmented reality display device of claim 2, where the twolight-redirection surfaces are curved.
 12. The augmented reality displaydevice of claim 11, where the two light-redirection surfaces areproduced via diamond turning.
 13. The augmented reality display deviceof claim 2, where one or both of the two light-redirection surfacesinclude Fresnel elements.
 14. The augmented reality display device ofclaim 1, where the aperture includes one or more annular rings formed inan environment-facing surface of the camera.
 15. The augmented realitydisplay device of claim 1, where the camera is positioned along anoptical axis extending away from the user eye.
 16. The augmented realitydisplay device of claim 1, further comprising a virtual image rendererconfigured to generate virtual images for display to the user eye viathe near-eye display, where the virtual images are superimposed over theimages depicting the real-world environment.
 17. The augmented realitydisplay device of claim 1, where the threshold is 50%.
 18. The augmentedreality display device of claim 1, where the threshold is 10% percent.